About

Sandeep Kumar Sahoo is an Associate System Engineer at IBM with 7.44 years of relevant experience in the field of Software Engineering, MapReduce, Apache Hive, Hadoop, HDFS, Java, and ETL. He completed his Electrical, Electronics and Communications Engineering degree from NIT Rourkela in 2015. Sandeep has extensive experience in working with highly unstructured and semi-structured data of 60 TB in size (180 TB with replication factor of 3) on a live 60 nodes Hadoop cluster running HDP 2.2. He has also worked on extracting data from Oracle Database into HDFS using Sqoop and created and worked on Sqoop jobs with incremental load to populate Hive External tables. He has developed Hive scripts for analyst requirements to perform ad hoc analysis and has a very good understanding of Partitioning and Bucketing concepts in Hive, designing both Managed and External tables in Hive to optimize performance. Sandeep has also solved performance issues in Hive and Pig scripts with an understanding of Joins, Group, and aggregation and how it translates to MapReduce jobs. He has experience in using Sequence files, RCFile, AVRO, and HAR file formats. Sandeep has also developed Oozie workflows for scheduling and orchestrating the ETL process. He has very good experience with both MapReduce 1 (Job Tracker) and MapReduce 2 (YARN) setups and monitoring and managing the Hadoop cluster using Ambari. Sandeep is also proficient in using Linux systems and has documented day-to-day tasks. Prior to his current role, he worked as an Associate Software Engineer at IBM, where he gained experience in Pig scripts to transform raw data from several data sources into forming baseline data. He also developed UDFs in Java as and when necessary to use in PIG and HIVE queries. Overall, Sandeep is a highly skilled and experienced Associate System Engineer with a strong background in Hadoop, Hive, ETL, and Java. He is committed to delivering high-quality results and is always looking for new challenges to expand his skillset.